A Comparison of Two Approaches For Selecting Covariance Structures in The Analysis of Repeated Measurements

The mixed model approach to the analysis of repeated measurements allows users to model the covariance structure of their data. That is, rather than using a univariate or a multivariate test statistic for analyzing effects, tests that assume a particular form for the covariance structure, the mixed model approach allows the data to determine the appropriate structure. Using the appropriate covariance structure should result in more powerful tests of the repeated measures effects according to advocates of the mixed model approach. SAS' (1996) mixed model program, PROC MIXED, provides users with two information criteria for selecting the `best' covariance structure, Akaike (1974) and Schwarz (1978). Our study compared these log likelihood tests to see how effective they would be for detecting various population covariance structures. In particular, the criteria were compared in unbalanced (across groups) nonspherical repeated measures designs having equal/unequal group sizes and covariance matrices when data were both normally and nonnormally distributed. The results indicate that neither criterion was effective in finding the correct structure. On average, for the 26 investigated distributions, the Akaike criterion only resulted in the correct structure being selected 47 percent of the time while the Schwarz criterion resulted in the correct structure being selected just 35 percent of the time. Not surprisingly, PROC MIXED default F-tests based on either of these selection criteria performed poorly according to results reported by the authors elsewhere.